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What I’m doing 

Estimate/predict/analyze human error in aviation 


Measures: Procedure Handling, Flight parameters, 
Communication, Teamwork, Situation Awareness,... 
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Teams 

• “a distinguishable set of two or more people who interact dynamically, 
interdependently, and adaptively toward a common and valued 
goal/object/mission, who have each been assigned specific roles or 
functions to perform, and who have a limited life span of membership” 
(Salas, Dickingson, Converse, &Tannenbaum, 1992) 

• Why do we need teams? 

• Tasks often too complex to be accomplishable by individuals; only 
by a team (Mathieu, Heffner, Goodwin, Salas, & Cannon-Bowers, 
2000; Cooke, Salas, Cannon-Bowers, & Stout, 2000) 

• Teams can share total workload and team members can monitor 
each other (Salas, et al., 1992) 

• Teams often achieve better performance than individuals (Lorge, 
Tuckman, Aikman, Spiegel, & Moss, 1955) 


09/30/2015 


- SAGA 2015 Bielefeld - 


Slide 4/ 22 


( Tffe 

c 





Lehrstuhl fur Ergonomie 

Technische Universitat Munchen 




- 


Crew Resource Management 

• Training process of teams to reduce human error (Helmreich, Merritt, & 
Wilhelm, 2009) 

• Origins in the early 1990s (NASA workshop) in the aviation domain 

• Meanwhile widely spread in different domains 


• Training content (European Aviation Safety Agency, 201 2) 

• Communication 

• Teamwork & Work Organisation 

• Situation Awareness & Decision Making 

• Current evaluation methods (IRR) 
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Situation Awareness 


“Situational awareness (SA) is the perception of environmental elements with 
respect to time or space, the comprehension of their meaning, and the 
projection of their status after some variable has changed, such as time, or 
some other variable, such as a predetermined event.” 


Task and Environmental 
Factors 

• workload 

• stressors 

• system design 

• complexity 


state of the 
environment/ 
system 


Situation Awareness 


Perception ^Comprehension 
of data and the of the meaning 
elements of the and significance 
environment of the situation 
(Level 1) J (Level 2) 


Projection 

of future states 
and events 

(Level 3) 




PERFORMANCE 
OF ACTION 
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Individual Factors 


goals • experience 

preconceptions • training 

knowledge . SAG'A 2d1'S s Bielefeld 


• Importance 
recognized by 
Endsley (1988) 

• Linked with 
performance 

• Majority of 
in/accidents as a 
consequence of 
lack in SA 

• Different models to 
describe SA 

According to Endsley (1993) 
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Situation Awareness of Teams 


• Shared SA important for teams 

• Level of sharedness is influenced by attention and mental models of the 
crew members (Bolstad, Cuevas, Gonzalez, & Schneider, XX) 

• Cooke (2004): teams have to solve problems, “detect and interpret cues 
as an integrated unit” 

Measurement of Team SA 

• Different approaches; more or less satisfying (Salmon et al., 2006) 

• Salmon et al. (2006) come to the conclusion that the „concept of team or 
shares SA requires much further investigation in itself, which in turn 
required the provision of reliable and valid measurement procedures" 

• Individuals SA 

• Degree of shared SA 

• Real time capable 

Simulator study with eye-tracking and communication (Dekker, 2002) 
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Situation Awareness 



Shared SA= (Team Member A) AND (Team Member B) 
Team Bandwidth = (Team Member A) OR (Team Member B) 
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But - Does that mean we are good if we only 
have a highly shared visual attention? 

...probably not... see Eastern-Air-Lines-Flight 401 

• Approach to Miami 

• gear lowering, but gear indication did not illuminate 

• Whole crew (three persons) worked on the problem (extremely high 
shared visual attention) 

• Captain leaned against yoke => AP disengaged => A/C lost altitude 

• Airplane crashed 

Nobody monitored any other parameter, all were focused on 
problem 

NTSB (1973): the landing gear indication “distracted the crew's 
attention from the instruments and allowed the descent to go 
unnoticed.” 

=> The low information bandwidth lead to the accident 
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Situation Awareness: Level 1 = perception 



Shared perception = (Team Member A) AND (Team Member B) 
Team Bandwidth = (Team Member A) OR (Team Member B) 
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Shared Visual Attention 
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Why Shared Visual Attention (SVA)? 


It might be... 

... that a high shared visual attention in some phases (e.g. before a 
decision making situation) is predicting good pilots' performance... 

... that in other situations, low shared attention combined with effective 
communication is better... 
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Measuring Shared Visual Attention 

• Gaze proportion (dwell time) on AOIs? 

• Number of fixations on AOIs? 

• Within a specific time slice 


=> Pearson correlation, moving average filtered (e.g. 750 frames = 30 sec.) 
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Shared Visual Attention 

• What do these results tell us? 

• Is r = .7 high in a cockpit environment? 

• Or just random, because pilots look at the same 
things at the same time anyway? 


=> Permutaton test (actual experimental data set) 

=^> Randomization test (random data) 
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SVA: Randomization & Monte Carlo Test 

• Taking all data 

• Differentiating between (all together) CPT and (all together) FO (but 
rather PF, PM) 

• Differentiating between CPT and FO of one crew 


=> Comparing those distributions might be interesting 

Are there crews, where the two pilots' overall gaze behavior is 
very similar? 

What leads to success? High correlation in overall behavior or in 
time-specific behavior? 

=^> Or variance? 
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SVA: Randomized real data (n=1 ,000,000 calculations) 
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SVA: Randomized real data 



and AOI2 at the same time? 
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SVA: Synchronized real data 



4000 


-1 - 0.8 - 0.6 - 0.4 - 0.2 0 02 0.4 0.6 
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cr 


• Different distribution 

• 28.74% of the pilots’ 
visual attention 
correlation values are 
statistically significant 
on the .95 level 
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SVA: Individual Distributions & time dependencies 


versus 


Crew 43; Windowsize 750 Frames 
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Team Bandwidth (ongoing analyses) 

Which / how many AOIs were checked by the crew (as 
one entity) within a specific time window? 


Example of one crew; window time = 30sec. 
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Next steps: 


Joint attention / RQA ? 


- Combining shared visual attention and team bandwidth 

- Relating both measurements to performance 
indicators 

Future 

- Real time application 

- E.g. more salient cues / adaptive displays in case 
crew members' shared visual attention is too low 
or in case crew's bandwidth is too low 
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Thanks to the two unknown reviewer! 

And thank you all for your attention (visual and auditory)! 


Questions? 

Comments? 

Ideas? 


Patrick Gontar 
gontar@tum.de 
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BACKUP 
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METHOD - Pilots & Scenario 

• 1 20 pilots holding valid ATPL with appropriate type ratings 

• Pilots were scheduled for participation as part of their working time 

• Participation in this full-flight simulator experiment was not voluntary 

Approach scenario: 

• Approaching New York (A 346, long 
fuel on board for 1 h 

• When lowering the gear, hydraulic 
system malfunctioned; 
nose gear not down and locked; 
fuel on board for 30min due to high 
aerodynamic drag 

• Go-around and procedures 


haul crews) on a visual approach; 
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METHOD - Scenario & Initial Rating 

• When extending the flaps, they jammed; 360 or aborting approach; 
fuel on board for about 15min; scenario for rating began here 

• FOR-DEC (on which they were trained on) or recognition primed 
decision, performing procedures 
and checklists, or aborting them 

• Landing 
Initial Rating: 

• Based on the instructor operating 
the simulator 

• Selecting four videos showing 
the same scenario with different performance levels (outstanding, 
medium-high, medium-low, poor) of the crew members 
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